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1 Introduction 



Let Xi,. . . ,Xn be independent and identically distributed M'^-valued random variables with un- 
known probability density /. The aim of this paper is to study the joint kernel estimation of the 

location 6 and of the size ^ = f{9) of the mode of /. The mode is assumed to be unique, that is, 
/(,t) < f{9) for any x ^ 9, and nondegcnerated, that is, the second order differential D"^ f{9) at the 
point 9 is nonsingular (in the sequel, D"''g will denote the differential of order m of a multivariate 
function g). 

The problem of estimating the location of the mode of a probability density was widely studied. 
Kernel methods were considered, among many others, by Parzen [18], Nadaraya [17], Van Ryzin 
[26], Riischendorf [23], Konakov [10], Samanta [24], Eddy ([5], [6]), Romano [20], Tsybakov [25], 
Vieu [27], Mokkadem and Pelletier [13], and Abraham et al. ([1], [2]). At our knowledge, the 
behaviour of estimators of the size of the mode has not been investigated in detail, whereas there 
are at least two statistical motivations for estimating this parameter. First, a use of an estimator 
of the size is necessary for the construction of confidence regions for the location of the mode (see, 
e.g., Romano [20]). As a more important motivation, let us underline that the high of the peak 
gives information on the shape of a density; from this point view, as suggested by Vieu [27], the 
location of the mode is more related to the shape of the derivative of /, whereas the size of the 
mode is more related to the shape of the density itself. Moreover, the knowledge of the size of the 
mode allows to measure the pertinence of the parameter location of the mode. 

Let us mention that, even if the problem of estimating the size of the mode was not investigated 
in the framework of density estimation, it was studied in the framework of regression estimation. 
Miiller [16] proves in particular the joint asymptotic normality and independence of kernel esti- 
mators of the location and of the size of the mode in the framework of nonparametric regression 
models with fixed design. In the framework of nonparametric regression with random design, a 
similar result is obtained by Ziegler ([32], [33]) for kernel estimators, and by Mokkadem and Pel- 
letier [14] for estimators issued from stochastic approximation methods. 

This paper is focused on semirecursive kernel estimators of 9 and f{9). To explain why we chose 
this option of semirecursive estimators, let us first recall that the (nonrecursive) wellknown kernel 
estimator of the location of the mode introduced by Parzen [18] is defined as a random variable 0* 
satisfying 

fniel) = sup ay), 

yes.'' 

where /* is Rosenblatt's estimator of /; more precisely, 

where the bandwidth (/i„) is a sequence of positive real numbers going to zero and the kernel is a 
continuous function satisfying lim||2,|[^_|_oo K{x) = 0, f^a K{x)dx = 1. The asymptotic behaviour of 
91 was widely studied (see, among others, [5], [6], [loj, [13], [17], [18], [20], [23], [24], [26], [27]), but, 
on a computational point of view, the estimator 0* has a main drawback: its update, from a sample 
size n to a sample size n -|- 1, is far from being immediate. Applying the stochastic approximation 
method, Tsybakov [25] introduced the recursive kernel estimator of 9 defined as 

r„.T.^,.,..J^w.(5^). 
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where Tq G is arbitrarily chosen, and the stepsize (7„) is a sequence of positive real numbers 
going to zero. The great property of this estimator is that its update is very rapid. Unfortunately, 
for reasons inherent to stochastic approximation algorithms properties, very strong assumptions 
on the density / must be required to ensure its consistency. A recursive version of Rosenblatt's 
density estimator was introduced by Wolverton and Wagner [30] (and discussed, among others, 
by Yamato [31], Davies [3], Dcvroye [4], Mcnon et al. [12], Wertz [29], Wegman and Davies [28], 
Roussas [22], and Mokkadem et al. [15]). Let us recall that is defined as 

Its update from a sample of size n to one of size n + 1 is immediate since clearly satisfies the 
recursive relation 

/.W=(l-i)/.-:W + 4A-(^). 

This property of rapid update of the density estimator is particularly important in the framework 
of mode estimation, since the number of points where / must be estimated is very large. We thus 

define a semirecursive version of Parzen's estimator of the location of the mode by using Wolvcrton- 
Wagner's recursive density estimator, rather than Rosenblatt's density estimator. More precisely, 
our estimator 0„ of the location 6 of the mode is a random variable satisfying 

fnien) = sup fn{y). (2) 

Let us mention that, in the same way as for Parzen's estimator, the fact that the kernel K is 
continuous and vanishing at infinity ensures that the choice of On as a random variable satisfying 
(2) can be made with the help of an order on W^. For example, one can consider the following 
lexicographic order: x < y if the first nonzero coordinate of x — y is negative. The definition 

en = inf<y e R'^ such that fn{y) = sup > , 

[ xGM'^ J 

where the infimum is taken with respect to the lexicographic order on W^, ensures the measurability 
of the kernel mode estimator. 

Let us also mention that, in order to make more rapid the computation of the kernel estimator 
of the location of the mode, Abraham et al. ([1], [2]) proposed the following alternative version of 
Parzen's estimator 

C = argmax/*(Xi). 

l<i<n 

Similarly, we could consider the following alternative version of our semirecursive estimator 

On = argmax/„(Xi). 

l<j<n 

However, to establish the asymptotic properties of 0*, Abraham et al. [2] prove the asymptotic 
proximity between ^* and 0*, which allows them to deduce the asymptotic weak behaviour of ^* 
from the one of 0*. In the same way, we can conjecture that the asymptotic weak behaviour of 
6n could be deduced from the one of On, but, in this paper, we limit ourselves on establishing the 
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asymptotic properties of On- 

Let us now come back to the problem of estimating the size f(9) of the mode. The ordinarily 
used estimator is defined as /i* = fn{(^n) ifn being Rosenblatt's density estimator and ^* Parzen's 
mode estimator); the consistency of ^* is sufficient to allow the construction of confidence regions 
for 6 (see, e.g., Romano [20]). Adapting the construction of /x* to the semirecursive framework 
would lead us to estimate f{6) by 

l^n = fn{On)- (3) 

However, this estimator has two main drawbacks (as well as jU*). First, the use of a higher order 
kernel K is necessary for {/in — fi) to satisfy a central limit theorem, and thus for the construction 
of confidence intervals of n (and of confidence regions for {0,ji)). Moreover, in the case when a 
higher order kernel is used, it is not possible to choose a bandwidth for which both estimators On 
and converge at the optimal rate. These constations lead us to use two different bandwidths, 
one for the estimation of 0, the other one for the estimation of ji. More precisely, let fn be the 
recursive kernel density estimator defined as 

where the bandwidth [hn) may be different from (/i„) used in the definition of /„ (see (1)); we 
estimate the size of the mode by 

An = fn{On), (4) 

where On is still defined by (2), and thus with the first bandwidth {hn)- 

The purpose of this paper is the study of the joint asymptotic behaviour of On and We first 
prove the strong consistency of both estimators. We then establish the joint weak convergence rate 
of On and fxn- We prove in particular that adequate choices of the bandwidths lead to the asymptotic 
normality and independence of these estimators, and that the use of different bandwidths allow 
to obtain sinuiltaneously the optimal convergence rate of both estimators. We then apply our 
weak convergence rate result to the construction of confidence regions for (^,/x), and illustrate 
this application with a simulations study. This application enlightens the advantage of using 
semirecursive estimators rather than nonrecursive estimators. It also shows how the estimation 
of the size of the mode gives information on the relevance of estimating its location. Finally, we 
establish the joint strong convergence rate of On and 

2 Assumptions and Main Results 

Throughout this paper, (/i„) and {hn) are defined as hn = h{n) and hn = h{n) for all n > 1, where 
h and h are two positive functions. 

2.1 Strong consistency 

The conditions we require for the strong consistency of On and /i„ are the following. 

(Al) i) K is an integrable, differentiable, and even function such that J^d K{z)dz = 1. 

ii) There exists C > such that J^^ \\^\\^\^{^)dz\ < oo. 

iii) K is Holder continuous. 

iv) There exists 7 > such that z 1— > \\z\\'~' \K{z)\ is a bounded function. 
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(A2) i) / is uniformly continuous on R'^. 

ii) There exists > such that J^a ||a^||^/(a^)'^a^ < oo. 

iii) There exists i] > such that z i-^ ||z||''/(z) is a bounded function. 

iv) There exists 9 eW^ such that f{x) < f{e) for all x^O. 

(A3) The functions h and h are locally bounded and vary regularly with exponent {—a) and (—a) 
respectively, where a g]0, l/{d + 4)[, d£ ]0, l/{d + 2)[. 

Remark 1 Note that (Al)iv) implies that K is hounded. 

Remark 2 The assumptions required on the probability density to establish the strong consistency 

of the semirecursive estimator of the location of the mode are slightly stronger than those needed 
for the nonrecursive estimator (see, e.g., [13], [20]), hut are much weaker than the ones needed for 
the recursive estimator (see [25]). 

Remark 3 Let us recall that a positive function (not necessarily monotone) C defined on ]0,oo[ 
is slowly varying if Vmit^oo C{tx) / C{t) = 1, and that a function G varies regularly with exponent 
p, p £ R, if and only if it is of the form G{x) = x^C{x) with L slowly varying (see, for example. 
Feller [8] page 275). Typical examples of regularly varying functions are , x^'logx, x'' log log a; , 
x'' log x/ log log and so on. 

Proposition 1 Let On arid fin be defined by (2) and (4), respectively. Under (Al)-(A3), 

lim 9n = a.s. and lim pn = p, a.s. 

n— ♦oo n-*oo 

2.2 Weak convergence rate 

In order to state the weak convergence rate of On and we need the following additional assump- 
tions on K and /. 

(A4) i) K is twice differentiable on W^. 

ii) z I— > zVK{z) is integrable. 

iii) For any {i,j) G {1, . . . d'^K/dxidxj is bounded integrable and Holder continuous. 

iv) ii' is a kernel of order q > 2 i.e. Vs G {1, . . . , g — 1}, V_7 G {!,..., d}, y^jK{y)dyj = 
and /iKd \yjK{y)\dy < oo. 

(A5) i) D'^f{0) is nonsingular. 

ii) D'^f is g-times differentiable, V/ and D'^f axe bounded. 

iii) For any (z,j) G {1, . . . sup^gj^d \\D^ (d"^ f /dxidxj) \\ < oo, and for any k E {1, . . . ,d}, 
sup^eRd \\D'iidf/dxk)\\ < oo. 

Remark 4 Note that (A4)ii) and (A4)iii) imply that VK is Lipschitz-continuous and integrable; 
it is thus straightforward to see that lim||3,||_^(^ ||V-fC(x)|| = (and in particular VK is bounded). 

We also need to add conditions on the bandwiths. Let us set 

^eiP') = n'^hn and C^j_{n) = nf^hn- 

(In view of (A3), Cq and are positive slowly varying functions, see Remark 3). In the statement 
of the the weak convergence rate of On and we shall refer to the following conditions. 
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(CI) One of the following two conditions is fulfilled. 

1 q a 1 — 2d 

i) 7 < a < -; — 7: 7: and - < a < — — — : 

W + 4 d+2q + 2 q d + 2' 

1 1,1 1 + dd 

< a < 7 and - — < a < 



' d+2q - d+A d + 2q + 2 2{d + 2) 

(C2) One of the following two conditions is fulfilled. 

1} < a < - — — and — < a < 



d + 2q 2 d + 2g + 2' 

ii) d = — — — , lim„_>oo = 00 and — — — — - < a < 



d + 2q' " ' 2{d + 2q) d + 2q + 2 

Remark 5 (CI) implies that limn^oo nhn'^'^'^^'^ = and lim.n^^ nhi^'^'^ = 0, whereas (C2) implies 
that lim^^oo nhf^'^'^'^'^ = 00 and lim„^oo nhf^'^'^ = 00. 

We finally need to introduce the following notation: 

B,{e)=[ "'(^f), with /3|= / y]K{y)dv, aq ^ I and a, / 1, (5) 



g!(l-ag) ^j=l f^j dx] 



^"l 1 r I mk<iK^{z)dz 

\ l+ad 



f{e)G 

" ; (6) 

l+dd 



G is the matrix dx d defined by G^*'-'^ = / — — (x)- — (x)dx, and, for any c, c > 0, -D(c, ( 

J^d dxi dxj 



Q ^ j where 1^ is the d x d identity matrix. 

Theorem 1 Let On and fin be defined by (2) and (4), respectively, and assume that (A1)-(A5) 
hold. 



i) If (CI) is satisfied, then 



^N{Q, AT.A). 



ii) If a = {d + 2q + 2) ^, d = {d + 2q) ^, and if there exist c, c > such that lim„_»oo nhfi^^'^'^^ = c 
and lim„_>oo nh^^'^ = c, then 



f Vnhi+^{en-e) 

Hi) If (C2) is satisfied, then 



^ N {D{c, c)ABq{e),AEA) 



h: 
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Remark 6 The simultaneous weak convergence rate of nonrecursive estimators of the location and 
size of the mode can be established by following the lines of the proof of Theorem 1. More precisely, 
set 



let 0* be Parzen's kernel estimator of the location of the mode and /i* = fni^^n) kernel 
estimator of the size of the mode defined with the help of 9^ and of Rosenblatt's density estimator 
f* (the bandwidth (hn) defining f* being eventually different from the banwidth (hn) used to define 
9^); Theorem 1 holds when 9n, fin, Bq{9), S are replaced by 9^, /x*, B*{9), S*, respectively. 

Part 1 and Part 2 in the case c = c = (respectively Part 3) of Theorem 1 correspond to the 
case when the bias (respectively the variances) of both estimators On and fin are negligeable in front 
of their respective variances (respectively bias). When c, c > 0, Part 2 of Theorem 1 corresponds 
to the case when the bias and the variance of each estimator 0„ and /i„ have the same convergence 
rate. Other possible conditions lead to different combinations; these ones have been omitted for 
sake of simplicity. 

Theorem 1 gives the joint weak convergence rate of On and fin- Of course, it is also possible 
to estimate the location and the size of the mode separately. Concerning the estimation of the 
location of the mode, let us enlighten that the advantage of the semirecursive estimator On on its 
nonrecursive version 0* is that its asymptotic variance \Y + a{d + 2)\~^ f{0)G is smaller than the one 
of Parzen's estimator, which equals f{0)G (sec, e.g. Romano [20] for the case d =1 and Mokkadcm 
and Pelletier [13] for the case d> 1)\ this advantage of semirecursive estimators will be discussed 
again in Section 2.3. The estimation of the size of the mode is of course not independent of the 
estimation of the location, since the estimator fin is constructed with the help of the estimator On- 
To get a good estimation of the size of the mode, it seems obvious that 0^ should be computed 
with a bandwidth (hn) leading to its optimal convergence rate (or, at least, to a convergence rate 
close to the optimal one). The main information given by Theorem 1 is that, for fin to converge at 
the optimal rate, the use of a second bandwidth (hn) is then necessary. 

Let us enlighten that, in the case when On and fin satisfy a central limit theorem (Parts 1 and 
2 of Theorem 1), these estimators are asymptotically independent, although, in its definition, the 
estimator of the size of the mode is heavily connected to the one of the location of the mode. As 
pointed out by a referee, this property was expected. As a matter of fact (and as mentioned in 
the introduction), the location of the mode is a parameter which gives information on the shape of 
the density derivative, whereas the size of the mode gives information on the shape of the density 
itself. This constatation must be related to the fact that the weak (and strong) convergence rate 
of On is given by the one of the gradient of /„, whereas the weak (and strong) convergence rate of 
fin is given by the one of /„ itself; the variance of the density estimators converging to zero faster 
than the one of the estimators of the density derivatives, the asymptotic independence of On and 
fin is completely explained. 

Let us finally say one word on our assumptions on the bandwidths. In the framework of 
nonrecursive estimation, there is no need to assume that {hn) and (hn) are regularly varying 
sequences. In the case of semirecursive estimation, this assumption can obviously not be omitted, 
since the exponents a and a stand in the expressions of the asymptotic bias Bq{0) and variance 
S. This might be seen as a slight inconvenient of semirecursive estimation; however, as it is 
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enlightened in the following section, it turns out to be an advantage, since the asymptotic variances 
of the semirecursive estimators are smaller than the ones of the nonrecursive estimators. 



2.3 Construction of confidence regions and simulations studies 

The application of Theorem 1 (and of Remark 6) allows the construction of confidence regions (si- 
multaneous or not) of the location and of the size of the mode, as well as confidence ellipsoids of the 
couple {9, ix). Hall [9] shows that, in order to construct confidence regions, avoiding bias estimation 
by a slight undersmoothing is more efficient than explicit bias correction. In the framework of 
undersmoothing, the asymptotic bias of the estimator is negligeable in front of its asymptotic vari- 
ance; according to the estimation by confidence regions point of view, the parameter to minimize 
is thus the asymptotic variance. Now, note that 

[l + a(d + 2)]-^/d 

[l + ad]~^ 

(where ATiA (respectively ATi* A) is the asymptotic covariance matrix of the semirecursive estima- 
tors {On, fin) (respectively of the nonrecursive estimators (6** , /i^)). In order to construct confidence 
regions for the location and/or size of the mode, it is thus much preferable to use semirecursive 
estimators rather than nonrecursive estimators. Simulations studies confirm this theoritical con- 
clusion, whatever the parameter (9, ji or {6,fJ,)) for which confidence regions are contracted is. For 
sake of succintness, we do not give all these simulations results here, but focuse on the construction 
of confidence ellipsoid for (0,/x); the aim of this example is of course to enlighten the advantage of 
using semirecursive estimators rather than nonrecursive estimators, but also to show how this con- 
fidence region gives informations on the shape of the density, and, consequently allows to measure 
the pertinence of the parameter location of the mode. 

To construct confidence regions for {9,n), we consider the case d= 1. The following corollary 
is a straightforward consequence of Theorem 1. 

Corollary 1 Let On and fin be defined by (2) and (4), respectively, and assume that (Al)-(A5) 
hold. Moreover, let [hn) and (hn) either satisfy (CI) or he such that Wvcin^oo nhi^^^ = and 
lim^^oo nh^^^ = with a = {2q + 3)~^ and a = {2q + We then have 

Moreover, (7) still holds when the parameters f{9) and f"{9) are replaced by consistent estimators. 

Remark 7 In view of Remark 6, in the case when the nonrecursive estimators 9^ and /i* are used, 
(7) becomes 

nhlymr ■(,;-,)^+ _ "t", f ^ (8) 



/(«)/,K«(x)<ic> " ' '/(«)/, if 2 W<ii 

(and, again, this convergence still holds when the parameters f{9) and f"{9) are replaced by con- 
sistent estimators). 
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Let (respectively /* ) be the recursive estimator (respectively the nonrecursive Rosenblatt's 
estimator) of /" computed with the help of a bandwidth /i„, and set 

_ {l + 3a)nhl[f:i{en)]'' „ _ il + a)nhn 



Moreover, let Ca be such that ¥{Z < c^) = 1—a, where Z is x^(2)-distributed; in view of Corollary 1 
and Remark 7, the sets 

fa = {{e,fl)/ Pn{en-Of + Qn{fin- fif <Cc,} 

s: = {{e,^i)/ p:{e*-ef + Q;,iil*,-^lf<c^} 

are confidence ellipsoids for {6, n) with asymptotic coverage level 1 — a. Let us dwell on the fact 
that both confidence regions have the same asymptotic level, but the lengths of the axes of the 
first one (constructed with the help of the semirecursive estimators On and /x„) are smaller than 
the ones of the second one (constructed with the help of the nonrecursive estimators 0* and /i*). 

We now present simulations results. In order to see the relationship between the shape of the 
confidence ellipsoids and the one of the density, the density / we consider is the density of the 
A^(0, (T^)-distribution, the parameter a taking the values 0.3, 0.4, 0.5, 0.7, 0.75, 1, 1.5, 2, and 2.5. 
We use the sample size n = 100 and the coverage level 1 — a = 95% (and thus c„ = 5.99). In each 
case, the number of simulations is iV = 5000. The kernel we use is the standard Gaussian density; 
the bandwidths are 

(log n) (log n) 

Table 1 below gives, for each value of cr, the empirical values of 9n , 6**, /In , fJ-n (with respect to 
the 5000 simulations), and: 

b the empirical length of the 0-axis of the confidence ellipsoid £^%; 
b* the empirical length of the ^-axis of the confidence ellipsoid 
a the empirical length of the /x-axis of the confidence ellipsoid £^%; 
a* the empirical length of the /x-axis of the confidence ellipsoid 6^^^; 
p the empirical coverage level of the confidence ellipsoid £5%; 
p* the empirical coverage level of the confidence ellipsoid S^y^ . 

Table 1 



a 


0.3 


0.4 


0.5 


0.7 


0.75 


1 


1.5 


2 


2.5 


On 


-0.002 


0.004 


0.001 


0.003 


0.002 


0.014 


-0.005 


-0.009 


0.014 


0*n 


0.003 


0.005 


0.001 


0.005 


-0.008 


0.016 


0.003 


-0.020 


-0.046 


b 


1.154 


1.346 


1.805 


2.898 


3.160 


5.218 


10.094 


17.866 


17.405 


b* 


1.166 


1.458 


1.968 


3.300 


3.582 


5.925 


12.943 


21.946 


23.715 


fJ-n 


1.335 


0.989 


0.782 


0.564 


0.522 


0.401 


0.263 


0.196 


0.155 




1.312 


0.979 


0.783 


0.562 


0.512 


0.388 


0.269 


0.193 


0.163 


a 


0.444 


0.399 


0.365 


0.322 


0.315 


0.283 


0.247 


0.224 


0.210 


a 


0.514 


0.459 


0.420 


0.369 


0.363 


0.327 


0.287 


0.261 


0.246 


P 


98.7% 


97.8% 


98.2% 


98.4% 


97.7% 


97.8% 


97.5% 


97.2% 


98.4% 


P 


98.6% 


98.1% 


98.4% 


98.2% 


96.8% 


96.6% 


96.9% 


97.7% 


98.2% 
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Confirming our theoritical results, we see that the empirical coverage levels of both confidence 
ellipsoids £*^o/^ and £"5% arc similar, but that the empirical areas of the ellipsoids £"5% (constructed 
with the help of the semirecursive estimators) are always smaller than the ones of the the ellipsoids 
£^t^^ (constructed with the help of the nonrecursive estimators). 

Let us now discuss the interest of the estimation of the size of the mode and the one of the joint 
estimation of the location and size of the mode. Both estimations give informations on the shape 
of the probability density and, consequently, allow to measure the pertinence of the parameter 
location of the mode. Of course, the parameter is significant only in the case when the high 
of the peak is large enough; since we consider here the example of the A^(0, cJ^)-distribution, this 
corresponds to the case when a is small enough. Estimating only the size of the mode gives a first 
idea of the shape of the density around the location of the mode (for instance, when the size is 
estimated around 0.16, it is clear that the density is very flat). Now, the shape of the confidence 
ellipsoids allows to get a more precise idea. As a matter of fact, for small values of cr, the length 
of the /Lt-axis is larger than the one of the ^-axis; as u increases, the length of the //-axis decreases, 
and the one of the 9-sxis increases (for a = 2.5, the length of the 0-axis is larger than 20 times the 
one of the //-axis). Let us underline that these variations of the lengths of the axes are not due 
to bad estimations results; Table 2 below gives the values of the lengths b (respectively b*) of the 
^-axis, a (respectively a*) of the //-axis of the ellipsoids computed with the semirecursive estima- 
tors 9n and /i„ (respectively with the nonrecursive estimators ^* and /i* ) in the case when the true 
values of the parameters f{6) and f"{9) are used (that is, by straightforwardly applying (7) and (8)). 



Table 2 

a 0.3 0.4 0.5 0.7 0.75 1 1.5 2 2.5 

b 0.159 0.327 0.571 1.357 1.572 3.227 8!895 18.260 31.899 

b* 0.190 0.390 0.682 1.622 1.879 3.858 10.631 21.825 38.127 

/X 1.333 0.998 0.798 0.570 0.532 0.399 0.266 0.199 0.159 

a 0.465 0.403 0.360 0.303 0.294 0.255 0.208 0.180 0.161 

a* 0.509 0.441 0.395 0.332 0.322 0.279 0.228 0.197 0.176 



2.4 Strong convergence rate 

To establish the joint strong convergence rate of 6n and we need the following additionnal 
assumption. 

(A6) i) h and h are differentiables, their derivatives vary regularly with exponent (—a — 1) and 
(—a — 1) respectively, 
ii) There exists uq G'N such that 

n> m> Uq =^ max • 



nhn^'^'^ ' nhn' j min {nhn^"^'^ ; nhn'} ' 



Remark 8 Assumption (A6)ii) holds when a a, and in the case a = a, it is satisfied when 

d 

j0.o{n) = {Cn{n))d'+2 for n large enough. 

Moreover, condition (C2) is replaced by the following one. 

(C'2) Either (C2) i) is fulfilled or a = - — — , lim„_oo = 00, and — — — <a< 

a + 2q 2 log log n 2(a-|-2g) 

1 

d + 2q + 2' 
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Before stating the almost sure convergence rate of {9'^,fln)'^, let us remark that Proposition 2.3 
in Mokkadem and Pelletier [13] ensures that the matrix G (and thus the matrix S) is nonsingular. 

Theorem 2 Let On and fin be defined by (2) and (4), respectively, and assume that (A1)-(A6) 
hold. 

i) If (CI) is fulfilled, then, with probability one, the sequence 



1 / ^/nhi+\en - 9) 



V21oglogn \^ ^Jnh^{f^n-n) J 
is relatively compact and its limit set is the ellipsoid 

£: = G M'^+^ such that u'^ A'^T.-^ A'^u < l| . 

i) If a = {d + 2q + 2)"^, a = {d+ 2q)~^, and if there exist c, c > such that 

lim„_>oo nhn'^^'^'^'^ / (2 log log n) = c and lim„_i.oo nh^'^'^ / (2 log log n) = c, then, with probability 
one, the sequence 

1 ( Vnh^+\9n -9)\ 



V21oglogn \^ ^J nhfXl^n - fi) J 
is relatively compact and its limit set is the ellipsoid 

S = ^ue W^-^^ such that (A" V - D (c, c) Bg{e)f S"^ {A-^u - D (c, c) Bq{e)) < l} . 



Hi) If (C'2) is satisfied, then 



Remark 9 (C'l) implies that Ynnn^^nhn ^ /log log n = and lim„^oo?^^n ^/loglogn = 0, 
whereas (C'2) implies that lim„_,.oo n/in^^*"*"^ /log log n = oo and lim„_»(X) n^n^^^ /log log n = oo. 

Laws of the iterated logarithm for Parzen's nonrecursive kernel mode estimator were established 
by Mokkadem and Pelletier [13]. The technics of demonstration used in the framework of nonre- 
cursive estimators are totally different from those employed to prove Theorem 2. This is due to 
the following fondamental difference between the nonrecursive estimator 0* and the semirccursive 
estimator 0^. the study of the asymptotic behaviour of 0* comes down to the one of a triangular 
sum of independent variables, whereas the study of the asymptotic behaviour of On reduces to the 
one of a sum of independent variables. Of course, this difference is not quite important for the 
study of the weak convergence rate. But, for the study of the strong convergence rate, it makes 
the case of the semirecursive estimation much easier than the case of the nonrecursive estimation. 
In particular, on the oppposite to the weak convergence rate, the joint strong convergence rate of 
the nonrecursive estimators 0* and /i* cannot be obtained by following the lines of the proof of 
Theorem 2, and remains an open question. 
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3 Proofs 



Let us first note that an important consequence of (A3) which will be used throughout the proofs 
is that 

1 " 1 
if/3a<l, then lim — ^V/if = ^. (9) 

Moreover, for all e > small enough, 



1=1 ^ 



(10) 



As a matter of fact: (i) if aq < 1, (10) follows easily from (9); (ii) if aq > 1, since J2i K summable, 

(10) holds; (iii) if aq = 1, since a{q — e) < 1, using (9) again, we have S^Li hf = 0{h'^^), and 
thus (10) follows. Of course (9) and (10) also hold when (/i„) and a are replaced by (hn) and a, 
respectively. 

Our proofs are now organized as follows. Section 3.1 is devoted to the proof of the strong 
consistency of 0„ and In Section 3.2, we give the convergence rate of the derivatives of /„. 
In Section 3.3, we show how the study of the joint weak and strong convergence rate of 6n and 
jln can be related to the one of Vfn{0) and /n(^)- In Section 3.4 (respectively in Section 3.5), we 
establish the joint weak convergence rate (respectively the joint strong convergence rate) of V/„(0) 
and fn{d)- Finally, Section 3.6 is devoted to the proof of Theorems 1 and 2. 

3.1 Proof of Proposition 1 

Since 9n is the mode of fn and 9 the mode of /, we have: 

< fie) - fiOn) = [f{e) - fn{On)] + [fn{0n) " /(^n)] < [1(0) " /„(^)] + [fn{0n) - f{0n)] 

< |/(^)-/n(e)| + |/n(^n)-/(^n)|<2||/„-/||oo. (H) 

The application of Theorem 5 in Mokkadem et al. [15] with |a;| = and Vn = logn ensures that for 
any (5 > 0, there exists c(5) > such that P[(logn)||/„-E(/„)||oo > < QW{-c{S)Y.7=ihf/{lognf). 
In view of (9), since ad < 1, we can write 

n^exp f-c(<5)pm') = n^exp ( -c{S)-^,^^] = o(l). 



oo 



(log n)^y \ (logn)^ n/i^ 

Borell-Cantelli's Lemma ensures that lim^^oo ||/n — lE(/„)||oo = a.s. Since lim„^oo ||IE(/n) — /||c 
0, it follows from (11) that lim„^oo/(^n) = 1(0) a-s. Since / is continuous, limy^n^jjo /(z) = and 
6 is the unique mode of /, we deduce that lim„^oo 0n = a.s. Now, we have 

I/in -l^\< \fn{0n) " f{0n)\ + |/(^n) " /(^)| < ||/n " /||oo + 2||/„ - /||oo, 

where the last inequality follows from (11). As previously, one can show that lim„_^oo ||/n~/||c!o = 
and thus lim„^cxD An = A* ^-s- 

3.2 Convergence rate of the derivatives of the density 

For any d-uplet [a] = (^ai, . . . ,0;^^ G N'', we set |a| = ai + ■ ■ ■ + and, for any function g, let 
d^"^g{x) = d^^'^g / {dx^^ . . . dx'^'^){x) denote the [a]-th partial derivative of g. 
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Lemma 1 Assume (A3)-(A5) hold. Let (gn) and be defined as follows: 



Qn = fn and bn = hn or 
gn = fn and bn = hn. 



(12) 



For \a\ G {0, 1, 2}, we have 



lim 



n 



n— »oo y^V' b- 



(x) 



where is defined in (5). Moreover, if we set Mg = sup^gjgd \\Did^"^f{x)\\, then 



lim 



n 



— v-n ,q sup 



Lemma 2 Let U be a compact set o/M'^ and assume that (Al)iii), (A3), (A4) and (A5)ii) hold. 
Let {gn) and {bn) be defined as in (12). Then, for all ^ > and \a\ = 1, 2, we have 



sup 



aM5„(x)-E(a["i5n(x)) 



= o 



(log n 



1 1+7 



-.n f^d+2\a\ 



V" b 
Z^i=l "i 



a.s. 



Lemma 1 is proved in Mokkadem et al. [15]. We now prove Lemma 2. Set Vn = E"=i 5^^^'"']^/^ 
[(logn)^'^'^]~^/^. Applying Proposition 3 in Mokkadem et al. [15], it holds that for any S > 0, there 
exists c((5) > such that 



SUpUn 



5["]5n(x)-E(5["]5n(x)) 



> s 



< exp I —c{d) 



n ,d+2\a\ ' 



2vl 



Since lim^^oo X^^Li '^O = oo wc have, for n large enough, c((5)^"^^ 6^'''^'"'/(2i;^) > 

2 log n, and Lemma 2 follows from the application of Borel-Cantelli's Lemma. 

3.3 Relationship between ((6>„ - 6>)^, (A„ - m))^ and ([V/„(6')]^, /„(6>) - f{e))^ 

By definition of On, we have V fn{On) = so that 

yfn{On) - Vfn{e) = -V fn{9). (13) 

For each iG{l,...,ci},a Taylor expansion applied to the real valued application dfn/dxi implies 
the existence of en{i) = {en\i), ■ ■ ■ ,e^\i))^ such that 



dfr 

dx. 



I \eii^{i) - < \e^ri'^{i) - Vj G {1, . . . ,4 . 

Define the d x d matrix Hn = {Hn'^^)i<ij<d by setting Hn'''^ = q^.q^^ i^nii))] Equation (13) can 
be then rewritten as Hn{On — 0) = — V/n(^). Now, set 



Rn = fn{dn) — fn{d). 



(14) 
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We can then write: 

[D^fie)]-'Hn{en-e) 



fn{0)-f{O) 



+ 







(15) 



Let J7 be a compact set of containing 0. The combination of Lemmas 1 and 2 with \a\ = 2, 
9n = fn and hn = hn ensures that for any 7 > and e > small enough, 



sup 

xeu 



o 



= o 



n t c(+4 



a.s. 



n 



^ +ht' + -\ =o(l) a.s. 



n. 



n 



(16) 



Since D^f is continuous in a neighbourhood of 6 and since lim„^oo = O a.s., (16) ensures that 
lim^^oo -f^n = D'^f{6) a.s. It follows that the weak and a.s. behaviours of {{9n — 6)'^, {fin — 1^))'^ 
are given by the one of the right-hand-sided term of (15). 

3.4 Weak convergence rate of ([V/„(0)]^, UiO) - f{e)f 

Let us at first assume that the following lemma holds. 

Lemma 3 Let Assumptions (Al)i), (Al)iv), (A3), (A4)i) and (A4)ii) hold. Then 



Wr,. = 



V 



V^^[Vfn{0)-K{Vfnie)) 

The application of Lemma 1 gives 



lim 

n^oo 



EH 



^{Ue))-f{e) 



(-1)" yrf Rld''f(n\ 
n\ 2^.7 = 1 Pj Sx« 



(17) 



1) If < 1 and < 1, by using (9), it is straightforward to see that 



lim 

n— >oo 



E{fn{e))-f{e) 

2) Let us now consider the case aq > 1 and dq> 1. Wc have 
^/nhi+^E{Vfn{e)) = 



(18) 



n 



n EILi hi 



with, in view of (10), for all e > small enough, 

i.d+2 Z^i=l f^i 



nh^+'' ^'=''~' = O (^„|{l-(«-^){d+2))„-a9+ae 



E(V/,(e)), 



oil) 



n 



Applying (17), it follows that lim^^co Vnh^^ECV fn{0)) = 0. Proceeding in the same way 
for E{fn{9)), we obtain 



lim 

n—^oo 



Vnhi+^E{vfn{e)) 



(19) 
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The combination of either (18) or (19) and of Lemma 3 gives the weak convergence rate of 

i[vfnier,fn{0)-mf-- 

• If (CI) holds, then 



V 



7V(0,S). 



(20) 



li a = {d+2q + 2) ^ , a = {d-\-2q) ^ , and if there exist c, c > such that hm^^oo n/in"''^'^'''^ = c 
and hm„_>oo nhf^'^'^ = c, then 



sfj4Cue)-f{e)) 



V 



AriD{c,c)Bg{e),j]) 



(21) 



• If (C2) holds, since aq < 1 and dq < 1, (9) implies that 

/ ]^V/„(0) \ 



Proof of Lemma 3 To prove Lemma 3, we first prove that 



lim E(WnWl^) 

n— >oo \ / 



and then check that (Wn) satisfies Lyapounov's condition. Set 

1 



k,n 



^k.n — 



1 



-2 



hk 



V nhn 



hk. 



and note that 



E 



Now, for any s, t G {1, . . . , d}, we have 



dK_ f e-Xk \ dK^ f e-Xk 

dxs \ hk ) dxt \ hk 



3K fO^y^ OKfB-y-.^^^^^ 



I —( 

7]Rd dxs V 



hk J dxt \ hk J 
= hif{9)Gs,t + o{hi), 



and since, E 



dK l e-Xk 

dxs \ hi, 



0{h'^), we deduce that 



(22) 



(23) 



E 



= fie)Ghi i + o(i 



(24) 
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which impUes that hm„_»oo Z]fc=i ^{Yk,vX^n) = + o,{d + 2)] ^G. In the same way, we have 

2n 



E 



K 



Q — 



E K 



hk 



= hifiO) [ K\z)dz 

JRd. 



1 + o(l) (25) 



and thus Hm^^oo ELi ^(^fc J = + "^]~^ /iRd Moreover, set /i^ 

have 



E 



- Xk\ ^ f 9 — Xk 



hk 



hk 



= hf £vK[^^z)K['i-z)fie-hlz)dz. 



Noting that fie - hlz) = f{e) + hlRki9,z) with |i?fe(0,z)| < ||V/||oo||^||, we get 



E 



VK 



9 — Xk 
hk 



K 



9 — Xk 
hk 



fie) [ VK(^z)K('Sz)dz + hl f VK(^z)K(^z)Rk{9,z)dz 
jRd ^hk ' ^hk ' JR'i ^hk ' ^hk ' 



Since the function z ^ ^ Ki^z^K^z is odd (in each coordinate), the first right-handed integral is 
zero, and, since h\ equals either hk or hk^ we get 



E 



9-Xk 
hk 



K 



9-Xk 
hk 



< hf^'^wvfiu 



\KUI \\z\\\\VK{z)\\dz + \\VKU f \\z\\\K{z)\d^=0(hf+'A 



We then deduce that 



E 



= O 



VK 



( 



e-Xv 



\ hk 



- E {vK ^ 



9 — Xk 
hk 



K 



9-Xk 
hk 



Xk 



hk 



d+l 



mm{hk,hk) 
and thus, in view of (9), 

n 

Y,^{Yk,nZk,n) =0 



+ 0{hihi] =0[K^ h 



'■k"'k 



-1 _ d+l 



'■k "'k 



(26) 



fc=i 



{nhn'^-^){nhn'') k=l 



d+l ^ 1-d 
2 



which concludes the proof of (23). Now we check that {Wn) satisfies the Lyapounov's condition. 
Set p > 2. Since K and are bounded and integrable, we have Jj^^ ||VK(2:)p(iz < oo and 
/]gd \K{zy^dz < oo. It follows that 



5^E(iin,„in = o 



k=\ 



= o 



{-d-l)p^d 
k 



k=l 



= o 



VK 



hk 



f{v)dy 



ht\=o{l), 

- y 



f{y)dy 
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which concludes the proof of Lemma 3. 



3.5 A.s. convergence rate of ([V/„(0)]^, U{d) - f{0)) 

Let us at first assume that the following lemma holds. 

Lemma 4 Let Assumptions (Al)i), (Al)iv), (A3), (A4)i), (A4)ii) and (A6) hold. With probability 
one, the sequence 



V21oglogn ^nhi [^{0) - 



is relatively compact and its limit set is £ = ^ W^'^'^ such that < l}. 

The combination of either (18) or (19) and of Lemma 4 gives the almost sure convergence rate of 

{Wfn{0)V,fn{e)-f{e)f: 

• If (CI) holds, then, with probability one, the sequence 

\A^V/„(0) 

'~fn{0) - f{0) 



\/2 log log n 

is relatively compact and its limit set is £^ = {i^ G 1^^^+^ such that v^Yr^v < l}. 



(27) 



If a = (d + 2g + 2) ^, a = ((i + 1q) ^, and if there exist c, c > such that lim„^oo n/in"*"^'"''^ 
(2 log log n) = c and lim„_>oo ra/in^^'^/ (2 log log n) = c, then with probability one, the sequence 



/ 



U0) - f{0) 



n 



-\/21oglogn 
is relatively compact and its limit set is 

f = G M'^+^ such that {ly - D{c,c)Bq{e)f J^'^ {u - D{c,c)Bq{e)) < l} . 

If (C'2) holds, then 

I ""U 



(28) 



a.s. 



^[/n(e)-m] 



B,{0). 



(29) 



We now prove Lemma 4. Set 

r = f{0) 



G 

J^,K^{z)dz 









1 



Qn 



V nh„ 



Vh^Id 

Vh^ 



let {sn) be a sequence of M'^+^-valued, independent and AA(0, r)-distributcd random vectors, and 
set Sn = Y^=iQk^k- III order to prove Lemma 4, we first establish the following Lemma 5 in 
Section 3.5.1, and then show in Section 3.5.2 how Lemma 4 can be deduced from Lemma 5. 

Lemma 5 Let Assumptions (Al)i), (Al)iv), (A3), (A4)i), (A4)ii) and (A6)ii) hold. With proba- 
bility one, the sequence (T^) = (S~^/^A„S'„/\/21oglogn) is relatively compact and its limit set is 
the unit ball Fd+i(0, 1) = e M'^'*'^ such that \\u\\2 < l}. 
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3.5.1 Proof of Lemma 5 



Set Bn = ¥.{SnS'^), let ||a;||2 (respectively |||^|||2) denote the euclidean norm (respectively the 
spectral norm) of the vector x (respectively of the matrix A). The application of Theorem 2 in 
Koval [11] ensures that 

, [|S-^/^A,„9„[b 

nmsup — 1^^^^^^^^^^^=^^^^^^^^^^^= < 1 a.s. 

^2 |||S-V2A„S„A„E-V2|||^loglog|||S„|||2 
Since lim^^oo A„5„A„ = S and log log |||-Bn||l2 ^ log log n, we deduce that 

limsup||T„||2 < 1 a.s. (30) 

n— >oo 

Thus, the sequence (T„) is relatively compact and its limit set U is included in ;Sd+i(0, 1). Now, 
set Sd+i = jfu G M*^"*"^, ||ui||2 = l|, and let us at first assume that 

ViuGiSd+i, limsupiu'^r^ > 1 a.s. (31) 

n— >oo 

The combination of (30) and (31) ensures that, with probability one, Ve > 0,Vno > 1, 3n > no such 
that w'^Tn > 1 - £ and ||r„||| < 1 + e. Noting that ||T„ - w\\l = ||r„||| + - 2w'^Tn, it follows 
that, with probability one, Ve > 0, Vno > 1, 3n > no such that ||r„ — it;||| < 1 + 6 + 1 — 2(1 — e) = Se. 
Thus, with probability one, S^-i-i C U. To deduce that Bd-{-i{0, 1) C U, wc introduce (e^), a sequence 
of real- valued, independent, and AA(0, l)-distributed random variables such that (e^) is independent 
of (£fe). Moreover, we set 



k=l 



We then note that the previous result applied to (T„) = (S~^/^ A„,S„/V2 log log n) ensures that, 
with probability one, 5^+2 = {w e ||w||2 = 1} is included in the limit set of r„. Now let 

vr : R^+^ — > M'^^^ be the projection map defined by tt{{xi, . . . ,Xd+2)'^) = {^2, ■ ■ ■ ,Xd+2)'^- We 
clearly have 7r{Sd+2) = ■6^+1(0, 1) and Tr(Tn) = Tn, and thus deduce that, with probability one, 
Bd+i{0, 1) is included in the limit set of T^. To conclude the proof of Lemma 5, it remains to prove 
(31). In fact, we shall prove that, 

\/w^0, limsup ^^^^^^^ > Vw^Su; a.s. (32) 
n^oo V 2 log log n 

Set Vn = m.m{[nhn^'^^^^Y/^;[nh-'^]^/^}, = VnW^ An and Vn = E(A„5„S'^A^) ; we follow a 
method used by Petrov [19] in the proof of his Theorems 7.1 and 7.2. Since lim = 00, Vr > 0, 

n— »oo 

there exists a non-decreasing sequence of integers n^ such that n^ — ^ 00 as A; — ^ 00 and < 
{1 + t)'' < Vn^, {k = 1,2,...). Since lim„_^oo Hi-i/^ = 1, we obtain ~ {1+t)'^. Moreover, we 
have 

Vn, - Vn,_, = Vn,{l - ^) ~ K,-^- (33) 
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X(n) = V2F„loglogy„, ^K) = J2(F„, - J log log(F„, - 



Set 

It follows from (33) that tp{nk) ~ T^/^x('^fc-i)- Then for any 7 g]0, 1[ and k sufficiently large, we 
have 

- An^Snk_^ > (1 - 7)■0(^^fc)) 

> P {An.Sn, > (1 - |)V'(nfc)) - P (^An,Sn,_, > TtM^ 

> P [Ar,,Sn, > (1 - ^)xK)) - P (An,Sn,_, > 2|^x(nfe-i)) . 
Since An^Sj^. is A/^(0, V^j.)-distributed, we have 

{AnuSnu > (1 - ^)x("fe)) = I ^ exp ("-^^ 

(35) 



(34) 



'(i-i)V2y„,iogiogy„^ 
> [logKj-(^+'^)(^-i)^ 



for every and sufficiently large k. Set = u^^u;^A„j^S„^^_^ A^^^u;; since An^Sn,^_^ is AA(0, K^)- 
distributed, we have 



3 V V, 



exp ( — — \dt 



Let Pmin{A) (respectively Pmax{A)) denote the smallest (respectively the largest) eigenvalue of a 
matrix A, set = A^iJ^A^, and note that 



V, 



n-k-i 



(36) 



with 



^Wfc-i III2 



Pmax{^nk^nk-i^"-k-i^nk-i^nk) ^ 

< Ills 

It follows from (9) and Assumption A6)ii) that 

,-(d+2) „ r-d 



A A A~^ 



A~^ A A A~-^ 



max 



(37) 



(38) 



^From (36), (37) and (38), we deduce that, for sufficiently large fc. 



Kit, 2p^aa;(E„j,_j^) 4p^a2;(E) 



and therefore, for sufficiently large A;, 



4 '1 > 

-^rik^rik-i ^ 



< 



1 /•'^ 



exp 



< [logT4,_J 36p^„.(i:, 



(39) 
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The inequalities (34), (35) and (39) imply that 



P [Ar^^Sn, - An,Sn^_^ > (1 - 7)V'(nfe)) > [\ogVn,]-^'+^^^'-^^ - [logF„,_J --^^^ . 
Thus, for sufficiently large k and r, there exists c > such that c does not depend on A; and 



> (1 - 7)V'("/c)) > c 
Choosing /x such that (1 + ^) (1 — 7/2)^ < 1, we get 



P {Ar^.Sn, - > (1 - 7)V(nfe)) > |/c-(^+'^)(^-i)' 

and thus J^k^i^^k'^n^. — An^.Sn^._-^ > — l)i^{nk)) = 00. Applying Borel-Cantelli's Lemma, we 
obtain 



P {Ar,,Sn, - An,Sn,_, > (1 " 7)V'(nfe) i-O.) = 1. 



(40) 



Now, 



lim sup 



l^"fc'^"fc-l I 
X{nk-i) 



< lim sup 



Vnk\\w\\2 



\^nk_iSny,_i II2 



/ 2vl^_^ {w'rAn,_,Bn,_,An^_,w) log log 



< lim sup ■ 

fe— ►oo 



\J'^'"lk-i (^^Su;) log log 



Applying Theorem 2 in Koval [11] again, and using the fact that lim„_>oo ^n-^n^n = ^1 we obtain 
limsup^^go ||An5n||2/-\/2 IIISIII2 log logn < 1 a.s. Therefore, 



lim sup 

fc— +00 



\An^Sn^_^ I 



< lim sup ■ 

X(^fc-l) jt-<-oo Vnk_iVw'^T,W 

< 2Vn^_JVn 

obtain limsup^^^o^^ |^„j.S'„j._ -^|/x(?^A;-l) < 2||tt;||2-^|||S|||2/Viy'^Sw a.s. Set e G]0, 1[ and 
K = 2\\w hVWh/VvF^. Noting that 

(1 - 7)V'(nfc) - 2Kx{nk-i) ~ [(1 - 7) V^(l + r)"'/' - 2k{1 + t)-V2] ^(rik), 

and noting that 7 can be chosen sufficiently small and r sufficiently large so that (1 — 7)\/t(1 + 
r)-V2 _ 2k(1 + r)-V2 > 1 - e, we obtain 

P {An.Sn, > (1 - £)x(nfe) i-O.) > P (An.Sn, > (1 - 7)V'("/c) - 2Kx(nfe_i) i.o.) . 

Taking (40) into account, we then obtain ¥ (Am^Sn^ > (1 — £)xi^k) i-o.) = 1- We thus get 
limsup„_>(^ A„S'„/x(n) > 1 a.s., which proves (32), and concludes the proof of Lemma 5. 



a.s. 



Since ||| A„^. A„^^_ JII2 = [/>max(A„^^_j A„j^ A„j^ A„^^_ J]^/^ < 2vn^_Jvn^, for sufficiently large k, we 
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3.5.2 Proof of Lemma 4 

Now, set 

Vk = 



-d/2 



-d/2 



VK 
K 



EiVK 
E{K 



-Xk 



(^)) 



and Tfc = E{VkV^). In view of (24), (25) and (26), we have limfc^oorfc = T. It follows that 
3A;o > 1 such that 'ik > ko, is inversible; without loss of generality, we assume ko = 1, and set 
Uk = T^^^'^Vk- Set p g]2, 4[ and let £ be a slowly varying function; we have: 



[k log log k)P/'^ 



O 



= o 



\\vk[ 



9-Xk 



k. 1 \\P 



K 



( e-Xk \ 
\ hk ) 



{k log log fe)p/2 



d-dp/2 _|_ yi-dp/2 



{k log log A;)p/2 } 
= O (£(fc) \^k-i^+r^-')i^-ad)] + ^-[i+(|-i)(i-ad)]J^ 

so that ^^.(/cloglog A;)^*'/^E(||[7,fc||'P) < oo. By application of Theorem 2 of Einmahl [7], we de- 
duce that X]fe=i ^A; ~ Z^fe=i % = o(v^nloglogn) a.s., where % are independent, and M{0,ld+i)- 
distributed random vectors. It follows that 



^ pi/2r-i/2y^ -J2ek = o(Vnloglogn) a.s. 



(41) 



k=l 



k=l 



Now, 



A. 



k=l k=l 

n 

k=i 



fc-i 



(with^ = 0) 



fc=i 

n-l 



fc=i 

n— 1 



An XI (^'fc ~ ^'fc+l) (V^log log k^ 



+ AnQn 



k=l 



o (^\/n log lognj 



a.s. 
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Moreover, 

n— 1 



An ^ (Qfe - Qfe+i) o {^^/k\og\ogk^ 

k=l 

/ I , d+2 1 / _d+2 _rf+2\ 

V V ' - ) o (V/^ log log /c) 







d _ d 

2 -/i 2 



V 







d+2 d+2 

2 -/l; 2 



'fc + l 



d+2 



Set = 2 and 0(s) = , and let G [/c, /c + 1]; since ^' and ^' vary regularly 

with exponent (a(d + 2)/2 — 1) and {a,d/2 — 1) respectively, we have 



n-l 



fc=i 



d+2 
2 



d+2 



/n-l 

vT) = o i;</)'(«fc) 



and 



n-l 



fc=l 



d _ d 

2 -/l 2 



"fc+l 



^=1 



/n-l 



0'(s)cis = O U. 



d+2 
2 



^fc=l 



O ^^'K) = o / ~4>'{s)ds ]=0[hn 



so t hat An Y JlJi {Qk - Qk+i) [o (V^ log log /e)] = o ( y^Iog log n) . Since A^Qn [o (V" log log n)] 
o ("v/log log n) , we deduce that 



An ELi Qfcrv^r, 1/2^, An ELi Qfc^fc 



o(l) a.s. 



V2 log log n \J2 log log n 

The application of Lemma 5 then ensures that, with probability one, the sequence (An ^^=1 QkS^^^"^ 
r^^^^l4/V2 log log n) is relatively compact and its limit set is £^ = {z/ G M'^+^ such that v^Yt'^v < 
1}. Since 



+ 



vl/2-p-l/2 
A- 



Vk 



-v/2 log log n 



\/21oglogn \/2 log log n 

with limfc^oo (-^rf+i — r^^^Tfe ^^^) ~ ^' L^nima 4 follows. 
3.6 Proof of Theorems 1 and 2 

In view of (15) (and the comment below), Theorem 1 (respectively Theorem 2) is a straightforward 
consequence of the combination of (20), (21) and (22) (respectively (27), (28) and (29)) together 
with the following lemma, which establishes that the residual term Rn (defined as in (14)) is 
negligeable. 

Lemma 6 Let Assumptions (Al)-(A5) hold. If (C2) holds, then lim„_>oo ^n^-^n = a.s. Other- 
wise, lim„_>oo nh^Rn = a.s. 
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Proof of Lemma 6 We first note that a Taylor's expansion implies the existence of such that 

llCn-^nll < ll^n-^ll and 

= {9n - 9f [V/„(Cn) - V/(Cn) + V/(Cn) - V/(0) 

Let V be a compact set that contains 0; for n large enough, we get 



\Rn\ 



o IK 



sup||V/n(x)-V/(a;)|| + ||Cn-e| 
.xev 



O pn - e\\ sup \\Vfn{x) - Vf{x)\\ + - 9\ 



On the one hand, let us recall that the a.s. convergence rate of (On — 0) is given by the one of 
[D'^f{e)]'^Vfn{0) (see (15) and the comment below). One can apply (27), (28), and (29) and 
obtain the exact a.s. convergence rate of 9n — 9. However, to avoid assuming (A6), we apply here 
Lemmas 1 and 2 (with |a| = 1 and {gn,bn) = {fmhn)), and get the following upper bound of the 
a.s. convergence rate of 9n — 9: for any 7 > and e > small enough, 



ll^n — ^11 

On the other hand, we have 



n, 



n 



a.s. 



(42) 



sup ||V/„(x) - V/(x)|| < sup ||V/n(x) - E (v/„(x)) II + sup ||E (v/n(x)) - V/C 



The application of Lemmas 1 and 2 with |a| = 1, {gn,bn) = {fn,hn) ensures that, for any 7 > 
and £ > small enough. 



sup||V/„(x)-V/(a;)||=o( I =0 



x€V \V nhi'^'^ ^ ) VV '^^^ 

Let C denotes a generic slowly varying function that may vary from line to line. 



(log»)^+^ , Iq-e 
d+2 "-n 



a.s. (43) 



Let us first assume that (CI) holds. The application of (42) and (43) ensures that for any 
e > small enough, 

V^ll^n - ^11 sup ||V/„(x) - V/(a;)|| = O {c{n) L-5(i-»(<i+2)-2S) + re-<i'^)\\+o{l) a.s. 
Observe that by (Cl)i), it is straightforward to see that 2a + a{d + 2) < 1 and a < a{q — e) 



for any e > small enough, so that y nhn\\9n — ^|| supj.gy ||V/n(x) — V/(a:)|| = o(l) a.s. 
Moreover, the application of (42) ensures that 



'nhi\\9n-9f = o(£(n) [n-5(i-M<^+2)+«<i)+n5(i-»'^-4«M)]) a.s. 

Now, by (Cl)ii) we have 2a{d + 2) — dd < 1 and dd+4a{q — e) > 1 for any £ > small enough, 
and thus it follows that ^ nh^\\9n — 9\\'^ = o(l) a.s., which ensures the first part of Lemma 6. 
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We now assume that (C2) holds. Since aq < q/{d + 2q) < 1, using (9), (42) and (43), we have 



^||e„-^||sup||V/„(x)-V/(a;) 



= o(c{n) 



n 



^ ^ a(d+2) ^ 5(d+2q+2) 



29+2) 1 , s , o(d+29+2) "l \ 

2 +n-2-°(g-^)+ ' 2 jj+o(l) a.s. (44) 



On the one hand, for any e > small enough, it is straightforward to see that condition (C2) 
implies the following inequalities: 



a{d + 2)+a{d + 2q + 2) <2 and a(d + 2 + 2g) < 1 + 2a{q - e), 

aq + a(d + 2) < 1 and dq < 2a{q — e). 



Therefore, it follows from (44) and (45) that 

}-\\en-0\\sup\\Vfn{x)-Vf{x) 



o(l) a.s. 



On the other hand, observe again that by (42) and (46), we have 



O (/:(n) J„-(l-a9-«(<i+2)) +„aq-2a(g-e)j^ ^ ^^^^ 

which concludes the proof of Lemma 6. 



a.s., 



(45) 
(46) 
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